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Tactical  Decision  Aid  codes  provide  field  prediction  of  maximum  range  for  FLIR  use  using  simplified  local  environmental 
parameter  input.  A  series  of  experimental  comparisons  at  sea  using  airborne  operational  FLIRs  with  an  instrumented 
ship  target  have  shown  poor  correlation  of  observed  range  with  prediction  for  detection  and  recognition.  Classification 
and  recognition  range  in  UFLR  are  found  to  be  highly  insensitive  to  radiosonde  atmospheric  profile  data  input.  Previous 
work  has  addressed  modeling  of  the  average  target  to  background  contrast  temperature  difference  and  atmospheric 
propagation  of  contrast.  This  paper  addresses  the  implementation  of  the  MDTD  and  MRTD  algorithms  in  the  code. 
Comparisons  are  presented  of  the  prediction  accuracy  of  the  UFLR  TDA  using  the  standard  Moser/Hepfer  algorithm 
and  an  adaptation  of  the  Johnson  criterion  used  in  the  NVEOL  Ratches  code.  For  the  limited  data  set  of  the  study  a 
reduction  of  RMS  prediction  error  is  achieved  using  the  NVEOL  algorithm. 


A  Tactical  Decision  Aid  (TDA)  code  such  as  the  Forward  Looking  InfraRed  range  prediction  codes  evaluated 
in  this  study  is  a  simplified  computer  model  designed  for  application,  for  example,  by  a  Navy  aircrew  at  pre-flight  briefing. 
It  consists  typically  of  (a)  a  target  model  giving  a  contrast  temperature  difference  (AT)  against  an  appropriate  background, 
(b)  a  propagation  model  to  project  this  contrast  from  the  target  location  to  the  sensor,  and  (c)  a  sensor  performance 
model  specifying  a  Minimum  Delectable  Temperature  Difference  (MDTD)  for  target  detection  or  a  Minimum  Resolvable 
Temperature  Difference  (MRTD  or  MRT)  for  classification  or  recognition.  Target  inputs  for  currently  operational  navy 
FLIR  TDAs  range  from  menu  selection  of  standard  target  type  and  size  to  a  complex  faceted  heat-balance  model 
providing  aspect-dependent  target  area  and  average  temperature.  The  propagation  of  the  apparent  contast  AT  to  the 
sensor  is  computed  typically  from  some  derivative  of  LOWTRAN  output  based  on  a  measured  marine  boundary  layer 
vertical  profile.  Maximum  range  is  determined  by  application  of  a  defined  apparent  AT  criterion  for  detection  or 
recognition  at  a  selected  probability  (  eg  50%).  TDAs  currently  in  use  in  the  Navy  include  the  FLIR  performance  module 
UFLR  incorporated  in  the  Tactical  Environmental  Support  System  (TESS)  and  the  EOTDA  (MKIH)  adapted  from  the 
Air  Force  for  naval  use. 

For  a  number  of  years  the  Naval  Academic  Center  for  Infrared  Technology  at  the  Naval  Postgraduate  School 
has  been  conducting  a  series  of  validation  measurements  in  cooperation  with  NRL  Monterey  (formerly  NOARL  West), 
using  the  135  foot,  294  too  Research  Vessel  POINT  SUR  as  an  instrumented  target  for  operational  FLIR-carrying  naval 
aircraft.  The  object  of  these  measurements  has  been  the  comparison  of  observed  with  predicted  ranges  under  conditions 
in  which  the  environmental  factors  are  accurately  known.  Scries  of  measurements  have  been  made  from  1986  to  1990, 
in  conjunction  with  a  variety  of  aircraft  (and  FLIR)  types.  The  summarized  observations  from  these  tests  and  predictions 
of  the  UFLR  code  have  been  compared  previously*'2.  Comparisons  have  been  made  using  the  UFLR  code,  the  EOTDA 
MK1I  and  MK1U,  and  in  all  cases  significant  discrepancies  have  been  found.  UFLR8,  a  modification  of  UFLR  to  include 
sky  reflection  and  real  target  dimensions  and  temperature  difference,  has  shown  only  minor  performance  improvement 
over  UFLR.  With  the  adaptation  of  the  EOTDA  to  naval  targets  further  comparisons  have  been  made3  still  without 
satisfactory  agreement.  The  objective  of  this  paper  is  to  '-onsider  the  detection  or  recognition  criterion  as  a  source  of 
error.  This  analysis  concentrates  on  data  taken  in  Monterey  Bay  during  the  measurement  period  2  to  7  November  1989, 
compared  with  the  TESS-UFLR  code. 
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2,  CHARACTERISTICS  QF  EXISTING  TDA  CODES, 

The  UFLR  operational  TDA  code4  used  for  the  analyses  in  this  paper  was  adapted  by  the  Naval  Environmental 
Prediction  Research  Facility  from  the  PREOS  code  developed  by  the  Naval  Ocean  Systems  Center  UFLR  is  used  in  two 
versions:  TESS-UFLR  incorporated  in  the  navy’s  Tactical  Environmental  Support  System  (TESS)  for  field  use,  and 
FNOC-UFLR,  a  central  site  version  used  by  Fleet  Numerical  Oceanography  Center  for  worldwide  prediction  based  on 
a  forecast  atmospheric  structure  data  set.  UFLR  provides  ranges  for  detection,  categorization  and  recognition  of  targets 
as  a  function  of  aircraft  altitude.  TESS-UFLR  uses  local  atmospheric  profile  data  from  the  Atmospheric  Environmental 
File  of  TESS,  and  requires  keyboard  input  of  surface  wind  speed  and  visibility.  Aircraft  or  FL1R  type  is  menu  selected; 
UFLR  supports  ail  navy  FLIRs.  Target  type  is  selected  from  a  menu  of  marine  targets,  each  represented  by  fused  effective 
dimensions  and  fixed  contrast  temperature.  Atmospheric  propagation  is  computed  with  a  parameterized  version  of 
LOWTRAN38  with  the  Kalz-Rhunke  aerosol  extinction  model,  and  a  layered  atmosphere  limited  by  the  resolution  of 
the  vertical  profile  data. 

The  MKIII  EOTOA  is  an  adaptation  of  the  Air  Force  Electro-Optical  TDA  developed  and  maintained  by  AFOL 
MKIII  supports  many  navy  FLIRS  including  that  used  in  this  study,  and  incorporates  a  ship  signature  model  (TCM2) 
developed  by  Georgia  Tech  Research  Institute.  TCM2  treats  internal  heat  sources  and  reflection  and  absorption  of  solar 
and  sky  radiation,  and  outputs  an  aspect-dependent  average  temperature  and  effective  area,  A  simple  sea  surface  radiance 
model  including  emitted  and  reflected  radiance  from  the  wind-ruffled  surface  is  used  for  target  thermal  contrast. 

3.  MEASUREMENT  PROCEDURES. 

The  R/V  POINT  SUR  is  a  research  ship  owned  by  the  National  Science  Foundation  and  operated  by  Moss 
Landing  Marine  Laboratories  for  a  consortium  of  West  Coast  institutions  including  NPS.  For  the  measurements  reported 
here  the  ship  was  equipped  with  an  array  of  thermistor  sensors  for  ship  skin  temperature  distribution  measurement,  and 
data  were  recorded  continuously  from  a  full  suite  of  meteorological  and  oceanographic  instruments.  Hourly  observations 
were  recorded  of  position,  ship  speed  and  heading,  surface  wind  speed  and  direction,  air  temperature,  pressure  and 
humidity,  cloud  cover,  precipitation,  visibility,  sea  state,  and  sea  surface  temperature.  Rawinsonde  probes  were  launched 
from  shipboard  to  profile  the  atmosphere  up  to  10,000  feet.  FLIR  range  observations  for  detection,  classification  and 
recognition  were  made  from  navy  Marine  Patrol  (MPA)  aircraft  during  seventeen  overflights  grouped  into  four  periods 
spread  over  a  week.  These  sorties  comprise  9  day  and  8  night  passes.  Only  the  detection  range  data  are  discussed  in  this 
paper. 


The  first  three  measurement  periods  experienced  weak  onshore  surface  wind  and  calm  seas,  with  weak  surface 
pressure  gradients  and  high  pressure  aloft.  The  boundary  layer  was  confined  to  about  1000  feet  by  the  subsidence 
inversion.  The  last  measurement  period  was  affected  by  the  passage  of  a  low  pressure  area,  with  increased  on-shore 
surface  winds  and  some  haze  and  low  stratus  clouds.  The  boundary  layer  extended  to  about  2600  feet. 

4,  RESULTS 

The  observed  and  computed  data  are  summarized  in  Table  1.  The  measurements  are  grouped  into  the  periods 
around  0600  GMT,  Nov  2,  1989  (2200  Local  Solar  Time,  (LST)  Nov  1)  and  1000  GMT  Nov.  2  (0200  LST.  2  Nov.),  0100 
GMT,  3  Nov  (1700  LST,  2  Nov)  and  2200  GMT  Nov  7  (  1400  LST,  7  Nov).  These  time  periods  are  represented  in  the 
following  figures  by  the  symbols  circles,  triangles,  squares  and  diamonds  respectively. 

The  observed  ranges  of  Table  1  are  shown  in  Figure  1  plotted  against  the  predictions  of  the  UFLR  and  EOTDA 
codes.  Figure  la  shows  predictions  using  the  standard  UFLR  code  with  the  'small  combattant*  target  input  from  the 
menu.  This  shows  a  large  degree  of  scatter;  the  linear  regression  line  drawn  indicates  significant  overprediction,  and  a 
coefficient  of  determination  r  of  0.44.  Figure  lb,  from  the  Mk  II  EOTDA  using  the  standard  UFLR  target  (since  Mkll 
does  not  internally  support  ship  targets)  surprisingly  shows  significant  underprediction,  but  much  less  vertical  scatter,  and 
an  i2  of  only  0.24.  Figure  1c  using  the  TCM2  model  specifically  of  the  R/V  POINT  SUR  in  the  MKIII  EOTDA  shows 
closer  numerical  agreement  but  an  i 2  of  only  0.07.  This  target  model  computes  effective  area  and  temperature  difference 
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for  the  real  target,  in  this  case  the  R/V  POINT  SUR.  These  low  correlations  lead  to  consideration  of  a  source  of 
discrepancy  other  than  the  target  contrast.  In  the  following  sections  the  criterion  for  detection  and  the  MDTD/MRTD 
are  considered. 


5.  ANALYSIS 


5.1  Criteria  for  Detection  and  Recognition  Range. 

The  UFLR  FUR  program  computes  the  maximum  range  for  detection,  categorization  and  recognition,  defined 
by  the  condition  that  the  perceived  target-to-background  temperature  difference  is  equal  to  the  system  Minimum 
Detectable  Temperature  Difference,  MDTD,  for  detection,  or  the  Minimum  Resolvable  Temperature  Difference  MRTD 
with  the  required  degree  of  resolution.  The  form  of  MDTD  adapted  from  Hepfer4  represents  'single  pixel"  or  'Star* 
detection. 

For  categorization  or  recognition  the  UFLR  code  incorporates  the  pixel  resolution  criterion  developed  by  Moser 
5,  incorporating  Hcpfer’s  modification  for  eye  spatial  filtering  for  small  targets.  This  describes  the  resolution  required  for 
a  given  level  of  discrimination  by  a  required  number  of  resolution  pixels  within  the  total  image  of  the  target,  described 
as  an  equivalent  square  area. 

The  application  of  MRT  and  MDT  models  to  discrimination  of  displayed  images  of  real  scene  objects  and  to 
range  prediction  has  been  discussed  by  Rosell s .  The  classic  Johnson  criterion7  for  visual  discrimination  replaces  the 
real  scene  with  a  bar  pattern  of  similar  contrast  to  the  real  target,  with  the  maximum  number  of  bars  resolvable  across 
the  minimum  target  dimension.  The  Ratches  (NVEOL) 4  code  incorporated  a  system  of  multiple  discrimination  levels 
defined  in  terms  of  lines  per  minimum  object  dimension,  distinguishing  three  different  discrimination  levels  for  detection, 
depending  on  background  clutter.  In  this  scheme,  the  lowest  level  detection  (level  0)  requires  1-3  lines  per  critical 
dimension.  In  this  study  the  applicability  of  higher  level  discrimination  detection  was  investigated  by  application  of  the 
NVL  three-bar  detection  criterion  to  range  prediction  in  UFLR. 

5.2  Ship  Range  Prediction  with  the  NVEOL  Criterion. 

The  basis  of  comparison  between  the  Moser/Hepfer  criterion  for  detection  or  recognition  as  implemented  in  the 
UFLR  code  and  the  Johnsoa  criteria  used  in  the  Ratches  (NVEOL)  code  is  the  detection  or  recognition  range.  Past 
experience  has  shown  that  the  standard  UFLR  small  combatant  target  is  a  poor  representation  of  the  R/V  POINT  SUR 
and  that  an  accurate  assessment  of  the  effective  target  size  and  apparent  AT  is  beneficial  for  TDA  comparisons.  An 
appropriate  model  for  target  calculation  is  the  Thermal  Contrast  Model  Version  2  (TCM2),  developed  at  the  Georgia 
Tech  Research  Institute  (GTRI),  and  implemented  in  the  Mark  III  EOTDA9.  A  stand-alone  version  of  TCM2  was 
executed  with  extensive  input  meteorological  data  sets  and  with  target/sensor  geometry  defined  by  the  observed  detection 
ranges  and  flight  profiles.  The  output  of  the  faceted  ship  model  was  reduced  to  the  effective  target  dimensions  and 
apparent  AT  between  the  target  and  the  sea  in  the  field  of  view  as  shown  in  Table  1.  The  GTRI  TCM2  model  utilizes 
a  simple  algorithm  to  account  for  the  variable  emissivity  of  the  wind-roughened  sea  surface  and  the  influence  of  reflected 
downdwelling  sky  radiation.  A  more  robust  model  of  the  1R  radiance  of  the  sea  surface  has  been  developed  by 
Wollenweber10  at  NOSC  which  utilizes  an  enhanced  version  of  LOWTRAN  6.  Apparent  AT  between  the  TCM2  ship 
target  and  the  NOSC  effective  sea  surface  temperature  (SST)  algorithm  are  also  shown  in  Table  1  as  well  as  ATs 
computed  between  the  TCM2  target  and  the  horizon  sky,  where  the  norizon  sky  was  modelled  as  a  biackbody  at  ambient 
temperature  11 . 

The  effective  ship  dimensions  and  the  three  sets  of  apparent  targct/backgroundATs  were  implemented  in  UFLR 
using  the  Moser/Hepfer  detection  criterion,  and  the  predicted  range  results  are  shown  in  Figures  2a- 2c  and  summarized 
in  Table  2.  Each  symbol  represents  a  different  measurement  period,  with  open  symbols  corresponding  to  daytime  and 
filled  corresponding  to  nighttime.  Comparison  between  these  figures  and  with  Figure  la  shows  that  a  more  accurate 
specification  of  effective  target  size  and  apparent  delta-T  improves  the  detection  range  predictions  from  11.7  nmi  RMS 
error  and  correlation  of  0.64  using  the  unmodified  UFLR  target  model  to  4.3  nmi  RMS  error  and  correlation  of  0.73 
using  the  TCM2  target  and  the  horizon  sky  as  background.  This  is  surprising  since  calculations  based  on  target  height 
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land  sensor  altitude  and  depression  angle  show  that  with  the  exception  of  a  single  case,  the  ship  target  at  detection  is 
completely  within  the  water  background  in  the  sensor’s  field  of  view.  The  corresponding  A  T  values  in  Table  1  show  that 
the  average  magnitude  of  the  apparent  AT  for  the  horizon  background  is  smallest,  2.34  versus  2,87°  C  for  the  GTRI 
model  and  3.26°  C  for  the  NOSC  mode;  the  standard  UFLR  ship  target  has  a  fixed  5.0  °C  apparent  AT. 

For  comparison  with  the  UFLR  detection  criterion, the  Johnson  deteclion/rccognition  criterion  was  adapted  from 
the  NVEOL  Ratcbes  FLIR  model  and  implemented  in  UFLR  in  place  of  the  standard  Moser/Hepfer  detection 
algorithm.  A  problem  has  been  known  to  exist  in  the  definition  of  the  critical  dimension  for  a  large  aspect-ratio  target. 

In  this  study  three  easily  implemented  forms  for  critical  dimension  have  been  used.  The  critical  dimension  for  detection 
in  UFLR  is  the  side  of  an  equivalent  square  target,  (xy),/3  ;  the  predicted  ranges  using  this  target  dimension  in  the 
NVEOL  algorithm  are  shown  in  Figures  3a  -  3c  for  the  same  three  apparent  AT  models.  These  ranges  arc  predicted  at 
the  50%  probability  of  detection  level.  The  results  show  less  scatter  than  for  the  Moser/Hepfer  criterion,  with  the 
smallest  RMS  error  ( 3.9  nmi)  and  the  largest  correlation  (.72)  associated  again  with  the  horizon  background.  From  Table 
2,  the  GTRI  SST  background  gives  the  worst  RMS  error  of  5.8  nmi  and  lowest  correlation  of  0.58  compared  with  the 
NOSC  SST  background  results  of  4.6  nmi  RMS  error  and  correlation  0.60. 

The  predictions  of  Figure  4  were  obtained  similarly  but  using  the  effective  horizontal  (x)  dimension  of  the  target 
as  critical  dimension  in  the  NVEOL  algorithm.  From  the  beam  aspect  most  detail  might  be  expected  in  this  dimension. 

For  each  of  the  sea  backgrounds  the  scatter  of  the  data  is  increased  with  a  resultant  increase  in  RMS  error  and  also  in 
correlation;  the  horizon  sky  background  however  shows  a  slight  decrease  in  correlation. 

For  the  predictions  of  Figure  5  the  vertical  (y)  dimension  of  the  target  (  fixed  Tor  the  R/V  POINT  SUR  at  8.8 
meters)  has  been  used  as  the  critical  dimension  for  detection.  This  is  the  smallest  of  the  three  dimensions  used  and  results 
in  the  shortest  ranges  and  the  smallest  RMS  errors.  From  Table  2  we  find  that  the  smallest  RMS  error  of  3.0  nmi  is 
associated  with  the  horizon  background,  which  also  has  a  correlation  of  0.72,  close  to  that  for  the  square  target  and 
horizon  sky  results  shown  in  Figure  3c. 

6.  CONCLUSIONS 

Previous  comparisons  of  observed  FLIR  detection  ranges  with  predictions  made  using  the  UFLR  model  have 
shown  considerable  scatter,  extreme  overprediction,  and  poor  correlation.  Previous  improvements  in  the 
target  /background  model  and  atmospheric  transmittance  model  have  yielded  marginal  increases  in  prediction  accuracy. 

The  latest  modification  of  UFLR  to  include  the  NVL  Ratches  model  implementation  of  the  Johnson  Criterion  has  shown 
additional  improvement  with  this  limited  data  set.  RMS  errors  have  been  reduced  from  nearly  12  nmi  to  under  3  nmi 
with  an  accompanying  small  increase  in  the  correlation  coefficient.  Most  importantly,  the  extreme  overpredictions  ! 

associated  with  UFLR  have  been  reduced.  Other  factors  which  may  affect  the  detectibility  of  targets  but  have  not  been 
considered  here  may  include  IR  sea  clutter,  white  caps,  and  sea  spray.  Human  factors  may  also  be  significant. 

Several  aspects  of  the  NVL  algorithm  should  be  evaluated  further  for  use  with  ship  targets.  It  is  not  clear  how  1 

the  critical  dimension  for  target  detection  should  be  defined,  taking  into  account  the  large  hull  form  and  sparse  i 

superstructure  and  aspect  ratio  change  with  viewing  angle.  Human  factors  and  the  appropriate  level  of  discrimination  for  , 

ship  detection  should  be  further  considered. 
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